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Amenn ments to the Claims: This listing of claims will replace all prior versions, an< 



of clair is in 



Listing of Claims: 

1. (Currently Amended) In a system using a plurality of fixed imagers covering a 
scene, fe method to create a high quality virtual image, in real-time, as seen from a vrHual 
viewpo nt of the scene, comprising the steps of; 

selecting at least two images corresponding to at least two of the plurai tjy of 
imagers to be used in creating the high quality virtual imag e each of the fee least 
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i mages corresponding to respective viewpoints different from the virtual vi 



Tiaps 



1) 



t wo images corresponding to respective viewpoints different from the virtual vie wpoint 



O 
maps 



the application 



imaccis 



creating at least two depth maps corresponding to the at least two i 
determining at least two sets of warp parameters using the at least two 
corresponding to said at least two images, each set of warp parameters 
rorrefsponding to warping one of the at least two images to the virtual viewpoir 
warping the at least two images to generate at least two warped image 
'eprdsenting the virtual viewpoint using the at least two sets of warp paramete^ 
:orrejsponding to said at least two images; and 

merging the at least two warped images to create the high quality vi 
■eprdsenting the virtual viewpoint . 



irtU'ii 



(Original) The method of claim 1 further comprising the step of selecting 
virtual \ iew^oint based on data supplied by an operator 



J« (Currently Amended) In a system using a plurality of fixed imagers covering a 
hod to create a high quality virtual image, in real-time, as seen from a viqtjual 
viewpoint of the scene, comprising the steps of; 

selecting the virtual viewpoint based on tracking at least one feature as the at 
ture moves within the scene; 

selecting at least two images corresponding to at least two of the plurali :y of 
imagers to be used in creating the high quality virtual imag e each of the i { feast 



creating at least two depth maps corresponding to the at least two imag 
determining at least two sets of warp parameters using the at least two 
corresponding to said at least two images, each set of warp parameters 
c orre^ponding to warping one of the at least two images to the virtual viewpoin 
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4. (Previously Presented) The method of claim 1 wherein the steps of cre< 
two depth maps comprises the steps of; 

bl) calculating a plurality of optical flow values between the at least two i 
calculating a plurality of parallax values corresponding to a plurality of 
coordinates in the at least two images from the plurality of optical flow values; 
calculating the at least two depth maps from the plurality of image coor|: 
:he plurality of parallax values 



b3) 
and 



depth naps 

*> 
he 

J>2) 



depth r laps 



e) warping the at least two images to generate at least two warped imagefe 
representing the virtual viewpoint using the at least two sets of warp parameters 
corresponding to said at least two images; and 

merging the at least two warped images to create the high quality virtual image 
3 the virtual viewpoint . 
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(Previously Presented) The method of claim 1 wherein creating the at Itfrst two 
comprises the steps of; 
mounting a plurality of depth sensing sensors viewing the scene colncid^ht with 
plurality of fixed imagers; 

selecting at least two depth sensing sensors corresponding to the at leafct two 
magjes; 

measuring a plurality of depth values corresponding to a plurality of imdde 
:oorc}inates in the at least two images with said at least two depth sensing sen ;f>rs; and 
creating the at least two depth maps from the plurality of depth values. 



(Previously Presented) The method of claim 1 wherein creating the at l^st two 
comprises the steps of; 
)1) separating the at least two images into a plurality of segments, pixels o^ |sach 
;egment having substantially homogenous values; 
>2) calculating a depth value corresponding to each segment; 
»3) optimizing the depth values corresponding to each segment; and 
>4) creating the at least two depth maps from the plurality of optimized depth values. 
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(Original) The method of claim 1 wherein the step of selecting the at l^st two 
selelcts exactly three images. 



(Original) The method of claim 1 further comprising the step of placing 
ilxed imagers in a geometric pattern. 
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(Original) The method of claim 1 wherein the step of selecting the at 

on a proximity of the virtual viewpoint to respective viewpoints corre 
two images. 



U ast 



(Original) The method of claim 1 wherein the step of selecting the at I^Jst two 
exactly two images. 



(Original) The method of claim 9 wherein the exactly three images correspond to 
magers from among the plurality of fixed imagers arranged in a triangle. 



(Currently Amended) A virtual camera system to create a high quality fiVtual 
l-time, as seen from a virtual viewpoint, comprising; 
plurality of fixed imagers; 
mag|e selection means for selecting an image from each of at least two of the illurality of 
cameras for use in creating the high quality virtual image each of the selqclted 



maoss qprresoondino to a respective viewpoint different from the virtual viewp oint 



estimation means for creating at least two depth maps corresponding to 
two images; 

fation means for calculating, based on the at least two depth maps, at 
warp parameters that define respective warpings of the at least two i 
rtual viewpoint; 

warper which applies the at least two sets of warp parameters from 
ation means to the at least two images respectively to create at least two 



of 



mage 



images; and 



;in irr age merger to merge the at least two warped images to generate the high 
nrtu4l image representing the virtual viewpoint . 
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view-b; sed 



L4 



color 



sensors 
least tv o 
sensors . 



L5. (Original) The system of claim 12 further comprising a plurality of dept i sensing 
alig led to view the scene coincident with the plurality of fixed imagers, where Dy the at 
depth maps are generated using data provided by the plurality of depth senshg 



16. (Withdrawn) A method which uses a plurality of fixed imagers to create 
one of < \ mosaic, a three dimensional model of a scene, and a virtual image, including 
of placi lg the plurality of fixed imagers in a hexagonal pattern, 



hexago 



neighbc ring 



neighbt 



in a din ctiori 
the plurality 



image 
pixels, c 



(Original) The system of claim 12 wherein the depth estimation means 
volumetric mapping means to create depth maps of the images. 



(Original) The system of claim 12 wherein the depth estimation means 
gmejitation depth calculation means to create depth maps of the images. 
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(Withdrawn) The method of claim 16, further including the step of depl^ing the 
pbttern of images on a planar surface. 



(Withdrawn) The method of claim 16 further including the step of depleting the 
hexagonal pattern of cameras on at least a portion of a tubular surface. 



(Withdrawn) The method of claim 16, wherein an angular separation b4itween a 
pair of the plurality of fixed cameras is less than 40°. 



(Withdrawn) The method of claim 19 wherein an angular separation be 
pair of the plurality of fixed cameras is greater than 30°. 



(Withdrawn) The method of claim 16 wherein the hexagonal pattern is blongated 
to provide an aspect ratio equal to an aspect ratio of an image produce fc^ one of 
of fixed imagers. 



(Withdrawn) A apparatus to create a local depth map of a scene from a 
Including a first plurality of pixels and a second image including a second plu 
ompising the steps of: 
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segmenting the first image into a plurality of contiguous segments, 
segmeht halving a further plurality of pixels of the first plurality of pixels, wherein pix^l values 
of the 1 urth<?r plurality of pixels are within a predetermined pixel value range; 

b. determining pixel depths of the further plurality of pixels by compar|r|ig the 
further plur ality of pixels to the second plurality of pixels; 

c. determining a segment surface based on the determined pixel deptl^ of the 
further|plur£|jty of pixels; and 

updating pixel depths of the further plurality of pixels to be within a 
predetdrmirled depth range of the segment surface by comparing the further plurality \\f pixels 
to the Jecorjd plurality of pixels. 



23. 



plane. 



methoc 



14. 
to 



showing the 

viewpoi nt, 
second 



includirg 
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second 



hypotht 



hypothetical 



(Withdrawn) The method of claim 22, wherein step (b) uses an optical 
cbmpare the given plurality of pixels to the second plurality of pixels. 
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(Withdrawn) The method of claim 22, wherein the given segment surface is a 



tie 



:;t image 
second 
ing a 



(Withdrawn) A method to create a local depth map of a scene from a fi 
scene from a first viewpoint and a second image showing the scene from 
first image including a first plurality of pixels and the second image incWd 
plurality of pixels, each pixel having a pixel value, comprising the steps of: 
dividing the pixel values into a plurality of pixel value ranges; 
segmenting the first image into a plurality of segments r each segme 
^ubset of pixels from the first plurality of pixels selected to have pixel valuta within 
urality of pixel value ranges; 

determining a plurality of depth values for the plurality of segments 
warping the first image into a warped image showing the scene from 
/iewfjoint using plurality of depth values; and 

calculating a first matching score of the second image and the warp4J image; 
hypothesizing that a given segment of the plurality of segments has 
depth value equal to a neighboring depth value of a neighboring segment 
calculating a hypothetical matching score of the second image and a 
warped image based on the hypothetical depth value; and 
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26. (Withdrawn) The method of claim 25, wherein step (c) includes the sfc ps of; 
d. determining pixel depths of a selected subset of pixels corresponding tc 
selected segment by comparing the selected subset of pixels to the second plurality o bixels; 

c2. . determining a selected segment surface based on the determined pixel pjepths of 
the sel ictec subset of pixels; 

c3. updating the determined pixel depths of the selected subset of pixels tc >e within 
a predetermined depth range of the selected segment surface by comparing the selected subset 
of pixe s to the second plurality of pixels; and 

c4. determining depth values for all pixels of the selected segment based o l the 
deternrpned pixel depths of the selected subsets of pixels of the selected segment surface 
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changing a given depth value of the given segment to the hypothet^jal depth 
hypothetical matching score is better than the first matching score. 



first 
a second 
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(Withdrawn) The method of claim 25, wherein step (g) includes the st4ps of; 
determining a first portion of the second image which corresponds to a 
:i^>n of the further segment in the warped image; 

determining a second portion of the second image which corresponds tc 
portion of the further segment in the hypothetical warped image; 

determining a changed portion of the second image which corresponds 
portion and the second portion; 

calculating a changed portion matching score of the changed portion of|the 
image and the warped image; 

calculating a changed portion hypothetical matching score of the seconc 
hypothetical warped image; and 

:alct lating the hypothetical matching score from the first matching score, the i^anged 
matting score, and the changed portion hypothetical matching score 



image 



7 of 13 



PAGE 9/15 * RCVD AT 1/2612006 5:47:10 PM [Eastern Standard Time] * SVR:USPTO-EFXRF-6/26 * DNlS:2738300 * CSID:61M070701 * DURATION (mm-ss):04-12 



